Selective enhancement of digital images

ABSTRACT

A method for image processing of a digital image is described comprising applying an image processing filter ( 17 ) as a function of the correspondence between each pixel in the image and a first target image characteristic ( 12 ) and a second target image characteristic ( 13 ). In a further embodiment, a method is described comprising applying an image processing filter as a function of the correspondence between each pixel, the received target image characteristic, and the input received from a user pointing device. A system and application user interface is also described.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. provisional application Ser.No. 60/456,150 filed Mar. 19, 2003, titled “System for Selective NoiseReduction and Enhancement of Digital Images.”

BACKGROUND

It is a well-known problem that noise in digital images is presentthroughout the image. While noise may appear more in certain attributesof a digital image, e.g., against sky, skin, background, etc., noise maynot be as visible when present against other detail types.

Currently available noise reduction processes address noise reductionfrom a global perspective (applying noise reduction to an entire image)often softening the image to an undesirable degree. Such problems existboth for luminance noise and chrominance noise. There are regions inimages (such as dark hair and shadows) where luminance noise does notdistract from the photographic qualities of the image and are often notperceived as noise. Chrominance noise, however, is more visible in thesame areas and must be reduced differently.

Most users of image editing applications face difficulties with“targeting” certain areas in an image. For example, a user who wants tosharpen the plant in the foreground of an image, but not the sky in thebackground of the image, faces a challenging task. In common imageediting applications, such as Adobe Photoshop®, the user would have tocreate a “selection” for the plant, before applying an image enhancementfilter, for instance, a sharpening filter. Typically, the user has to“draw” the selection using a pointing device, such as a computer mouse,around the plant. Only after creating such a selection, can the usersharpen the plant.

Further, the user often wants to sharpen the plant to a high degree andthe background to a lower degree. To do so, the user would first have toselect the plant, sharpen it to a high degree, then select everythingelse but the plant, and sharpen this to a lower degree. In anotherexample, given the case that there is a person in the given image andthe user wants to sharpen the plants in the image to a high extent, thebackground to a low extent, and the hair and the skin of the person inthe image to a medium extent, using selections with conventionalapplications becomes a highly challenging task.

Selecting an area in an image is a difficult task Therefore, imageediting applications such as Adobe Photoshop® offer a variety ofdifferent selection methods, all of which have a steep learning curve.What is needed is a method and system to make selective enhancement animage easier, and which would be applicable for all types of imageenhancement filters, such as sharpening, noise reduction, contrastchanges, conversion to black and white, color enhancement etc. Such amethod and system would provide for a range of image enhancements on aselective basis. Preferably, such a method and system would be able toprocess a digital image by applying an image processing filter as afunction of multiple image characteristics, or as a function of an imagecharacteristic and the input from a user pointing device.

SUMMARY

The disclosed method and system meets this need by providing for a rangeof image enhancements on a selective basis. The method and system isable to process a digital image by applying an image processing filteras a function of multiple target image characteristics, or in a furtherembodiment, as a function of target image characteristic and the inputfrom a user input device.

A method for image processing of a digital image comprising pixelshaving characteristics is disclosed, comprising applying an imageprocessing filter as a function of the correspondence between each pixeland a first target image characteristic and a second target imagecharacteristic.

A method for image processing of a digital image comprising pixelshaving characteristics is disclosed, comprising the steps of providingan image processing filter, receiving first target imagecharacteristics, receiving second target image characteristics,determining for each pixel to be processed, the correspondence betweenthe characteristics of that pixel and the first target imagecharacteristics and second target image characteristics and processingthe digital image by applying the image processing filter as a functionof the determined correspondence between each pixel and the first targetimage characteristics and second target image characteristics. Invarious embodiments, the image processing filter may be, for example, anoise reduction filter, a sharpening filter, or a color change filter.

In a further embodiment, an adjustment parameter may be received, andthen the application of the image processing filter is also a functionof the adjustment parameter. In various embodiments the adjustmentparameter may be an opacity parameter or a luminosity parameter.

In still further embodiments a graphic user interface may be providedfor receiving the first target image characteristics, the second targetimage characteristics, and optionally the adjustment parameter. Thegraphic user interface for receiving the adjustment parameter optionallymay comprise a slider.

In various embodiments the first target image characteristics, or thesecond target image characteristics, may be an image coordinate, acolor, or an image structure, and indicia may be used to representtarget image characteristics.

In a still further embodiment, the graphic user interface comprises atool to determine the pixel characteristics of an image pixel.

In a further embodiment, a camera-specific default settings areprovided.

An application program interface is disclosed, embodied on acomputer-readable medium for execution on a computer for imageprocessing of a digital image, the digital image comprising pixelshaving characteristics, comprising a first interface to receive firsttarget image characteristics, a second interface to receive secondtarget image characteristics, a third interface to receive a firstadjustment parameter corresponding to the first target imagecharacteristics, and a fourth interface to receive a second adjustmentparameter corresponding to the second target image characteristics.Optionally, a fifth interface comprising indicia representing the firsttarget image characteristics, and a sixth interface comprising indiciarepresenting the second target image characteristics, may be added. Atool to determine the pixel characteristics of an image pixel may alsobe added to the interface, and optionally, the third interface and thefourth interface may each comprise a slider.

A system for image processing of a digital image is disclosed, thedigital image comprising pixels having characteristics, comprising aprocessor, a memory in communication with the processor, and a computerreadable medium in communication with the processor, the computerreadable medium having contents for causing the processor to perform thesteps of receiving first target image characteristics, receiving secondtarget image characteristics, determining for each pixel to beprocessed, the correspondence between the characteristics of that pixeland the first target image characteristics and second target imagecharacteristics, and processing the digital image by applying the imageprocessing filter as a function of the determined correspondence betweeneach pixel and the first target image characteristics and second targetimage characteristics.

Optionally, the computer readable medium further has contents forcausing the processor to perform the steps of receiving a firstadjustment parameter corresponding to the first target imagecharacteristics and receiving a second adjustment parametercorresponding to the second target image characteristics. In a furtherembodiment, he system of claim further comprises a set ofcamera-specific default instructions embodied on a computer-readablemedium for execution on a computer.

A set of camera-specific default instructions embodied on acomputer-readable medium is disclosed, for execution on a computer forimage processing of a digital image, using one of the embodiments of themethod of the invention. The set of camera-specific default instructionsmay set the state of the application program interface.

A method for image processing of a digital image comprising pixelshaving characteristics is disclosed, comprising applying an imageprocessing filter as a function of the correspondence between eachpixel, the received target image characteristic, and the input receivedfrom a use pointing device.

A method for image processing of a digital image comprising pixelshaving characteristics is disclosed, comprising the steps of providingan image processing filter, receiving a target image characteristic,receiving a coordinate from a user pointing device, determining for eachpixel to be processed, the correspondence between the characteristics ofthat pixel, the target image characteristics, and the receivedcoordinates, and processing the digital image by applying the imageprocessing filter as a function of the determined correspondence betweeneach pixel, the target image characteristic, and the receivedcoordinates. In various embodiments the image processing filter may be,for example, a noise reduction filter, a sharpening filter, or a colorchange filter. A graphic user interface for receiving the target imagecharacteristic may be used, and optionally the graphic user interfacemay comprise indicia representing the target image characteristic.Example target image characteristics include an image coordinate, acolor, or an image structure.

An application program interface embodied on a computer-readable mediumfor execution on a computer for image processing of a digital image isdisclosed, the digital image comprising pixels having characteristics,comprising a first interface to receive a target image characteristic;and a second interface to receive a coordinate from a user pointingdevice.

A system for image processing of a digital image is disclosed, thedigital image comprising pixels having characteristics, comprising aprocessor, a memory in communication with the processor, a user pointingdevice, and a computer readable medium in communication with theprocessor, the computer readable medium having contents for causing theprocessor to perform the steps of receiving a target imagecharacteristic, receiving coordinates from a user pointing device,determining for each pixel to be processed, the correspondence betweenthe characteristics of that pixel, the target image characteristics, andthe received coordinates, and processing the digital image by applyingthe image processing filter as a function of the determinedcorrespondence between each pixel, the target image characteristic andreceived coordinates.

DRAWINGS

These and other features, aspects, and advantages of the presentinvention will become better understood with reference to the followingdescription, appended claims, and accompanying drawings, where:

FIG. 1 is a depiction one embodiment of an application user interfacesuitable for use according to the invention.

FIG. 2 is a depiction another embodiment of an application userinterface suitable for use according to the invention.

FIG. 3 is a depiction one embodiment of an application user interfacesuitable for use according to a further embodiment of the invention.

FIG. 4 is a depiction of a user interface showing application of theinvention.

FIG. 5 is a pictorial diagram of components usable with the system forenhancing digital images according to the present invention.

FIG. 6 is a pictorial diagram of the image sources useable for acquiringa digital image to be enhanced according to the present invention.

FIG. 7 is a block diagram of an embodiment of the method of theinvention.

FIG. 8 is a block diagram of a further embodiment of the method of theinvention.

FIG. 9 is a block diagram of an embodiment of the system of theinvention.

FIG. 10 is a block diagram of a further embodiment of the system of theinvention.

DETAILED DESCRIPTION

The method and program interface of the present invention is useable asa plug-in supplemental program, as an independent module that may beintegrated into any commercially available image processing program suchas Adobe Photoshop®, or into any image processing device that is capableof modifying and displaying an image, such as a color copier or a selfservice photo print kiosk, as a dynamic library file or similar modulethat may be implemented into other software programs whereby imagemeasurement and modification may be useful, or as a stand alone softwareprogram. These are all examples, without limitation, of image processingof a digital image. Although embodiments of the invention which adjustcolor, contrast, noise reduction, and sharpening are described, thepresent invention is useful for altering any attribute or feature of thedigital image.

Furthermore, it will become clear with regard to the current inventionthat the user interface for the current invention may have variousembodiments, which will become clear later in this disclosure.

The present invention is also useable with a method and systemincorporating user definable image reference points, as disclosed inU.S. Pub. No. US 2003-0099411 A1, Ser. No. 10/280,897, for “UserDefinable Image Reference Points”, which disclosure is expresslyincorporated herein by reference.

The Application Program Interface

The present invention, in its various embodiments, permits the selectionof areas of a digital image for enhancement. In preferred embodiments, auser interface component is present. Those skilled in the art will findthat multiple methods or implementations of a user interface are usefulwith regard to the current invention.

In one preferred embodiment of a user interface useable with the presentinvention, the interface allows the user to set a variety of types ofimage modifications in an image, which can be shown as graphic sliders,as shown in FIG. 1. The sliders could be implemented in a window whichfloats above the image, as will be evident to those skilled in the artwith reference to this disclosure. In one preferred embodiment, withreference to FIG. 2, the sliders are implemented in a window containingzoom enabled previews of the image, before and after application of theimage enhancement. In the embodiment shown in FIG. 2, a plurality ofsliders are available, so that the chosen image enhancement can operateas a function of these multiple inputs.

In another embodiment, with reference to FIG. 3, a plurality of imagecharacteristics are listed, and the user may choose to apply the chosenimage enhancement (noise reduction in the case of FIG. 3) to the areaselected. For example, by choosing “skin” from the table menu, the usercan paint on the noise reduction filter, and only skin areas will bemodified. In the optional further embodiment shown, erase, fill, andclear operations are available.

The application program interface is embodied on a computer-readablemedium for execution on a computer for image processing of a digitalimage. The interface receives the characteristics of the image which theuser desires to select. In a further embodiment, a second interfacereceives an image editing function assigned by the user.

Selective Enhancement Using a Selective Application Matrix

With reference to FIGS. 1 and 2, the plurality of sliders and graphicicons are inputs to a matrix, which for convenience we can describe as aSelective Application Matrix, abbreviated to SAM As will be evident tothose skilled in the art, other types of controllers are also possibleas inputs to the SAM. There are at least two, and typically five ormore, SAM controllers.

Preferably, the SAM controllers are displayed next to the image, andeach SAM controller is linked to a region in the image. The regions maybe described in a variety of ways. In one preferred method the regionsare described by image feature; for example, the first SAM controllermay be linked to sky, and the second may be linked to grass (not shown).

As is evident from FIG. 1 and FIG. 2, the SAM controller may have anassociated numerical input interface to set an adjustment parameter forfilter opacity, strength, or other variable. In a preferred embodiment aslider is used, but direct input or other interfaces are possible. Inthe previous sky/grass example, if the user sets the first SAMcontroller adjustment parameter to 80% and the second controller is setto 20%, the selected filter will be applied to 80% strength to the skyand to 20% strength to the grass. If the filter is a sharpening filter,the sly would be sharpened to 80% and the grass to 20%. The same wouldoccur for a filter that increases the saturation, reduces noise, orenhances the contrast. As a further example, the filter could be afilter that turns a color image into a black and white image, where thesliders would control the tonality in the image, so that in the blackand white image the sky would have an 80% tonality (dark) and the grasswould have a 20% tonality (being bright).

The SAM may be used for the purposes of noise reduction, imagesharpening, or any other image enhancement, where it is desired to beable to selectively apply the image enhancement.

With reference to FIG. 1, each SAM controller in that embodiment isrepresented by a set of icons and a slider for the adjustment parameter.Each of the SAM controllers is accompanied by one or more fields (1.1,1.2 and 1.3) that can represent target image characteristics. In FIG. 1,icon 1.1 represents a color, icon 1.2 represents an image structure, andicon 1.3 holds an image coordinate. In one embodiment, the color can bea RGB value, a structure can be a value derived from the difference ofadjacent pixels (such as the mean luminosity difference of horizontallyadjacent pixels, or local wavelet, or Fourier components), and an imagecoordinate could be an X and a Y coordinate.

If the first slider is supposed to be “linked” with the sky (how theuser creates such a “link” will be described below), then the color icon1.1 would contain a color that represents the sky (saturated blue), thestructure field would contain data that represents the structure of sky(a very plain structure), and the coordinate field would represent alocation somewhere in the sky (top of the image). The same principleapplies for the second SAM controller, which may, for example, be linkedto the “grass” (green, high detail structure, bottom of image).

The user can either set these values in icons 1.1 through 1.3 manually(such as by clicking on the icon and then selecting a color or astructure from a palette, or by entering the value via the keyboard), orthe user can use the eyedropper (see icon 1.5 in FIG. 1). Once the userclicks on the eyedropper, he can then click in the image. Once he clicksin the image, the software will then read the color, structure and thecoordinate, and fill these values into the icons 1.1 to 1.3. Optionally,as shown a check box 1.6 can be provided to select or deselect an givenSAM controller.

Not all embodiments require all of the icons 1.1, 1.2, and 1.3; at leastone of them is sufficient. For example, in FIG. 4, each SAM controllercomprises one icon and one slider for a parameter adjustment.

Any user control that enables the user to define a value can be used.This could be a field where the user can enter a number via thekeyboard, a wheel that can be rotated like a volume control on anamplifier, or other implementations.

With reference to FIG. 7, a digital image can then be processed usingmethod 10:

-   -   11) provide an image processing filter 17;    -   12) receive first target image characteristics;    -   13) receive second target image characteristics;    -   14) determine for each pixel to be processed, the correspondence        between the characteristics 16 of that pixel and the first        target image characteristics and second target image        characteristics; and    -   15) process the digital image by applying the image processing        filter as a function of the determined correspondence between        each pixel and the first target image characteristics and second        target image characteristics.

In one embodiment, for each pixel to be processed, the SAM controllerwhose characteristics match the given pixel best is determined, andusing that controller's values as inputs for the filter, the pixel ismodified.

In a further embodiment, a step can be added to receive 19 an adjustmentparameter and apply the filter 17 as a function of the adjustmentparameter. In a still further embodiment, camera-specific defaultsettings are provided 21 as described herein.

For example, where the user wants to sharpen a plant with 80% strengthand the sky in the background with 20% strength, this algorithm wouldidentify some pixels in the image to match the characteristics of theSAM controller set to the plant and sharpen those pixels with 80%. Otherpixels would be identified to match the SAM controller set to the skyand would then be sharpened with 20%, and still others might notidentify with either and might not be sharpened.

In order to avoid harsh transitions, definable image reference pointscould be used to allow for soft transitions from one area to another, asdisclosed in U.S. Pub. No. US 2003-0099411 A1, Ser. No. 10/280,897, for“User Definable Image Reference Points.” (That disclosure is expresslyincorporated herein.) This would be preferred for filters that changeluminosity or color, as the soft transitions provide a higher imagequality. In filters such as noise reduction or sharpening, speed ofprocessing may be more important.

The SAM can be used in many different ways. The filter can be any imageenhancement, and the values of the adjustment parameter can be anydominant parameter of that filter. The filters can be color enhancement,noise reduction, sharpening, blurring, or other filter, and the valuesof the adjustment parameter can control the opacity, the saturation, orthe radius used in the filter.

In still further embodiments, the filters can be a conversion to blackand white or a filter that raises the contrast. In such a filter theuser may want to make certain areas a little darker while applying thefilter, while brightening other areas. The SAM would then be implementedin a way that the value provided for each pixel in the named algorithmis used to darken or lighten the pixel to a certain extent.

Any filter known in the field of image editing, and any parameter ofthat filter can be controlled by a SAM.

Calculating A Selective Application Matrix

As an example of how the application user interface can be used with afilter will be described. In this embodiment, with reference to FIG. 1,the user can click on one of the icons representing target imagecharacteristics, such as color icon 1.1, and redefine the color that isassociated with the associated slider 1.4. In the following equation,these n colors will be referred to as C₁ . . . C_(n). The setting of aslider (i.e., the desired noise reduction for the color of the slider)will be referred to as S₁ . . . S_(n). It is preferable to normalize S₁. . . S_(n) so that it ranges from 0.0 to 1.0, where 1.0 represents 100%noise reduction.

The desired value S_(xy) can be calculated for each pixel in the imageas follows:$S_{xy} = {\sum\limits_{i = 1}^{n}\frac{S_{i}{V\left( {{{C_{i,1} - C_{{Ixy},1}}} + \ldots + {{C_{i,m} - C_{{Ixy},m}}}} \right)}}{\sum\limits_{u = 1}^{n}{V\left( {{{C_{u,1} - C_{{Ixy},1}}} + \ldots + {{C_{u,m} - C_{{Ixy},m}}}} \right)}}}$

Where:

S_(xy) is the value to be calculated for each pixel xy in the image I,ranging from MIN to MAX, to represent for example the opacity of a noisereduction algorithm applied.

n is the amount of sliders that are offered, such as 3 in the givenexamples.

m is the amount of target image characteristics that are used in theprocess.

V is an inversion function, such as V(x)=1/x, e^(−x) ² , 1/x², etc.

S_(i) is the value of the i-th slider, ranging from MIN to MAX.

C_(i,j) and C_(Ixy,j) are characteristics of a pixel or a slider,C_(i,j) being the j^(th) characteristics of the i^(th) slider, C_(Ixy,j)being the j^(th) characteristic of the pixel I_(xy).

The characteristics C can be directly derived from the values receivedfrom the target image characteristic icons 1.1, 1.2, and 1.3 as shown inFIG. 1. If the coordinates icon 1.3 is provided, the list ofcharacteristics C_(i,I) . . . . C_(i,j) will at least include one targetimage characteristic for the horizontal, and one target imagecharacteristic for the vertical coordinate. If a color icon 1.1 or astructure icon 1.2 is provided, additional characteristics will bederived from those fields. Note: To implement a SAM, not allcharacteristic fields 1.1, 1.2, or 1.3, as shown in FIG. 1, arerequired.

This principle can be used for filters like sharpening, noise reduction,color warming, and other filters where it is desirable to control theopacity of one filter.

The SAM can also be used to provide advanced input parameters to afilter. If a filter F′ has one parameter z that the user may want tovary throughout the image, such as I′_(xy)=F′ (I,x,y,z), this parameterz can be replaced with S_(xy) in order to vary the effect of the filterF′.

Such a filter F′ could be a blurring effect, and the parameter z couldbe a radius. In that case, the sliders would probably reach from 0.0(MIN) to, for instance, 4.0 (MAX), so that Ss is a radius between 0.0and 4.0. The blurring filter F(I,x, y,S_(wy)) would then blur the pixelsof the image depending on the variable S_(xy), which varies from pixelto pixel. With this technique, the user can blur the image withdifferent radii at different areas. For example, if there were only twosliders and the user “linked” one slider to the sky and set its value to3.5, and if the user “linked” the second slider with the face in theforeground and set its value to 0.5, the filter would blur the sky witha radius of 3.5, the face with a radius of 0.5, and other parts of theimage with varying radii between 0.5 and 3.5.

Another example for such a filter F′ could be any complex image filterwith many parameters in addition to z, such as a conversion to black andwhite, a relief effect, a painterly effect, an increase of contrast,etc. Many of such artistic or photographic filters often create “falloff areas” or “blown out areas.” A “fall off area” is an area in theimage that is completely black (large area of zero values) after thefilter is applied, and a “blown out area” is an area that is purelywhite. Neither effect is wanted. For instance, if the filter applies abrightening effect, areas that were “almost white” before filtering mayeasily become pure white after filtering. In such case it is desirablethat this area be darkened while filtering. This could be done, forinstance, by setting the lowest possible setting of the n sliders W) toa negative value and the highest possible setting of the n sliders(MAX).to the same positive value, such as −50 and 50, so that S_(xy)varies from −50 to 50 for each pixel on the image. The user couldconnect one of the sliders to that area that was almost white beforefiltering, and set the sliders value to below zero. The filter F′(I, x,y, z) would then receive a low value for z in this area and thereforelower the luminosity in this area while applying the filter. Thoseskilled in the art will be familiar with how to include z into thisprocess. For example, z may be simply added to the luminosity before anyfurther filtering takes place.

FIG. 4 shows a sample use of a SAM implementation used to prevent blownout areas during the image editing process. FIG. 4 (top) shows the imagewithout the SAM being used and FIG. 4 (bottom) shows the image with theSAM used to prevent the blown out effect.

Using the SAM for Camera-Specific Noise Reduction

The SAM can be combined with camera-specific noise reduction filters toprovide optimized noise reduction and increased control. If thiscombination is desired, the implementation of the sliders in FIG. 1 canbe camera specific. For example, a camera with a uniform noise behaviormay require fewer sliders (for example n=3) while a camera that producesnoise that is more structure dependent, relative to other cameras, mayrequire a larger number of sliders (for example n=8).

In a further embodiment of the invention, the default settings of thesliders could be made camera-specific. If the camera has a tendency toproduce excessive noise in blue areas of an image, the SAM might includea slider with a color field, which is set by default to blue and aslider value which is set by default to a high setting. Animplementation for a specific camera is shown in FIG. 2.

Noise and Detail Specific Tools

The use of detail-specific noise reduction and detail enhancement toolsare provided in one embodiment of the current invention allowing usersto use conventional pointing devices, such as a computer mouse or apressure sensitive graphics tablet and pen, to apply the prescribedtool. Current applications only allow users to brush-in effects in animage such as a fixed color, a darkening or a lightening effect, asharpening or a blurring effect.

With reference to FIG. 3, one embodiment of the current inventionprovides detail specific filters that focus on individual types ofdetail in order to protect specific details in the noise reductionprocess. By focusing on specific details that occur in most images, aspecific process can be created for selective noise reduction thatconsiders specific detail types. A variety of detail specific noisereducers can be designed, such as one designed for sky details,background details, skin details, and shadow details, for example. Thenoise reduction filter (in other embodiments other filters could beused) can then be brushed-in using a user pointing device 36.

With reference to FIG. 8, a digital image can then be processed bymethod 20:

11′) provide an image processing filter 17′;

12′) receive a target image characteristic;

18) receive a coordinate from a user pointing device 36;

14′) determine for each pixel to be processed, the correspondencebetween the characteristics 16′ of that pixel, the target imagecharacteristic, and the received coordinates.

15′) process the digital image by applying the image processing filter17′ as a function of the determined correspondence between each pixelthe target image characteristic, and the received coordinates.

Creating Noise Brushes for Different Image Structures and Details

In order to create a detail-specific noise reduction filter, a generalnoise reduction algorithm is required which differentiates betweenchrominance and luminance and different frequencies. For example, afilter could have one parameter for small noise, for noise ofintermediate sizes, and for large noise. If a filter based on a Laplacepyramid, Wavelets, or Fourier analysis is used, those skilled in the artwill know how to create a noise reduction filter that differentiatesbetween various frequencies/bands. The filter may also accept differentparameters for the luminance noise reduction strength versus chrominancenoise reduction strength. If this is done, the filter will be able toaccept a few different parameters: TABLE 1 High Frequencies/ MediumFreq./ Low Freq./ Luminance Luminance Luminance High Freq./ MediumFreq./ Low Freq./ Chrominance Chrominance Chrominance

For best results, locate a suitable combination of such parameters.

It is possible to correlate these target image characteristics tospecific enhancement algorithms using heuristic methods. For example,using a plurality of images, select-one image structure type, such assky, skin, or background. Using trial and error, experiment withdifferent values for the noise reducer on all of the images to determinethe optimal combination for the noise reduction for this structure type.For example, for the structure type background, the following-parametersmight be suitable: TABLE 2 100% 100% 100% 100% 100% 100%

Since the background of an image is typically out-of-focus and thereforeblurry, it is acceptable to reduce both chrominance and luminance noiseto a strong degree. On the other hand, the structure type sky might havethe following parameters: TABLE 3  25%  50%  75% 100% 100% 100%

This combination would be suitable as sky often contains very fine clouddetails. To maintain these details, the first table entry (highfrequencies/luminance) is set to 25% only. However, as sky consistsmostly of very large areas, it is important that the low frequencies arereduced to a rather large extent, so that the sky does not contain anylarge irregularities. Because of this, the third table entry is set to75%. The lower three table entries, which cover the chrominance noise,are all set to 100%, as sky has a rather uniformly blue color, againstwhich color irregularities can be seen very well.

Treating Chrominance and Luminance Noise

One embodiment of the current invention provides a range of options foroptimally reducing chrominance noise (noise that consists of some degreeof color) and luminance noise (noise with no appearance of color) in adigital image. The system described employs a range of techniques whileusing an approach that splits the image into one luminance channel (C1)and two chrominance channels (C2 and C3). The process of splitting thechrominance information from the luminance information in the image maybe performed in a constant fashion or using a camera-dependentimplementation.

Splitting the Image in Chrominance and Luminance

To gain the channels C₁, C₂, and C₃, the image can be transformed eitherinto “Lab” or “YCrCb” mode, or in an individual fashion, where C₁ couldbe calculated as x₁r+x₂g+x₃b, all x being positive. While doing so, itis important that a set of x₁ . . . x₃ is found which leads to a channelC₁ that contains the least possible chrominance noise. To do so, take animage containing a significant amount of chrominance noise and find aset of x₁ . . . x₃ where the grayscale image C₁ has the least noise.Finding the set of x₁ . . . x₃ with trial and error is an appropriateapproach To obtain the image channels C₂ and C₃, two further triples ofnumbers y₁ . . . y₃ and z₁ . . . z₃ are required, where all three setsmust be linear independent. If the matrix [x, y, z] were lineardependent it would not be possible to regain the original image colorsout of the information C₁ . . . C₃ after the noise reduction wereperformed. Find values for y₁ . . . y₃ and z₁ . . . z₃ so that theresulting channels C₂ and C₃ contain the least luminance information(the image should not look like a grayscale version of the original) andthe most chrominance noise (the color structures of the original shouldmanifest themselves as a grayscale pattern of maximal contrast in thechannels C₂ and C₃). The two triples (−1,1,0) and (0,−1,−1) are goodvalues to start with. If the user interface or system involves a stepthat requests information from the user on what digital camera/digitalchip/recording process is used, it may preferable to adjust the threetriples x₁ . . . x₃ . . . z₁ . . . z₃ based on the camera If a cameraproduces a predominant amount of noise in the blue channel, it may bepreferable to set x₃ to a low value. If it has the most noise in the redchannel, for instance with multiple-sensor-per-pixel chips, it may makesense to set x₁<x₃.

System

Preferably, the invention will be embodied in a computer program (notshown) either by coding in a high level language, or by preparing afilter which is complied and available as an adjunct to an imageprocessing program. For example, in a preferred embodiment, the SAM iscompiled into a plug-in filter that can operate within third party imageprocessing programs, such as Photoshop®. It could also be implemented ina stand alone program, or in hardware, such as digital cameras.

Any currently existing or future developed computer readable mediumsuitable for storing data can be used to store the programs embodyingthe afore-described methods and algorithms, including, but not limitedto hard drives, floppy disks, digital tape, flash cards, compact discs,and DVDs. The computer readable medium can comprise more than onedevice, such as two linked hard drives. This invention is not limited tothe particular hardware used herein, and any hardware presently existingor developed in the future that permits image processing can be used.

With reference to FIG. 9, one embodiment of a system 100 of the presentinvention comprises a processor 102, a memory 104 in communication withthe processor 102, and a computer readable medium 106 in communicationwith the processor 102, having contents for causing the processor 102 toperform the steps of one of the embodiments of the method 10 of FIG. 7.With reference to FIG. 10, a further embodiment of a system 200 of thepresent invention comprises a processor 102, a memory 104 incommunication with the processor 102, a user pointing device 36, and acomputer readable medium 106 in communication with the processor 102,having contents for causing the processor 102 to perform the steps ofone of the embodiments of the method 20 of FIG. 8.

With reference to FIG. 5 and FIG. 6, one hardware configuration useableto practice various embodiments of the method of the invention comprisesa computer monitor 32 and computer CPU 34 comprising processor 102 andmemory 104, program instructions on computer readable medium 106 forexecuting one of the embodiments of method 10 or method 20 on a digitalimage 38, for output on one or more than one printer type 42, or adigital display device 30 through the Internet. In at least oneembodiment a user-pointing device 36 provides coordinate information toCPU 34. Various pointing devices could be used, including pens, mice,etc. As will be evident to those skilled in the art with reference tothis disclosure, various combinations of printer type 42 or digitaldisplay device 30 will be possible.

Digital image 38 could be obtained from various image sources 52,including but not limited to film 54 scanned through a film scanner 56,a digital camera 58, or a hard image 60 scanned through an image scanner62. It would be possible to combine various components, for example,integrating computer monitor 32 and computer CPU 34 with digital camera58, film scanner 56, or image scanner 62.

In one embodiment, it is possible to have the program instructions querythe components of the system, including but not limited to any imageprocessing program being used, or printer being used, to determinedefault settings for such programs and devices, and use those parametersas the inputs into the SAM. These parameters may automatically bedetermined without operator intervention, and set as the defaults forthe system. Depending upon the particular needs, these defaults may befurther changeable by operator intervention, or not.

It is to be understood that in this disclosure a reference to receivingparameters includes such automated receiving means and is not to belimited to receiving by operator input. The receiving of parameters willtherefore be accomplished by a module, which may be a combination ofsoftware and hardware, to receive the parameters either by operatorinput, by way of example through a digital display device 32 interface,by automatic determination of defaults as described, or by acombination.

The enhanced digital image is then stored in a memory block in a datastorage device within computer CPU 34 and may be printed on one or moreprinters, transmitted over the Internet, or stored for later printing.

In the foregoing specification, the invention has been described withreference to specific embodiments thereof. It will, however, be evidentthat various modifications and changes may be made thereto withoutdeparting from the broader spirit and scope of the invention. Thespecification and drawing are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It should be appreciatedthat the present invention should not be construed as limited by suchembodiments, but rather construed according to the below claims.

All features disclosed in the specification, including the claims,abstract, and drawings, and all the steps in any method or processdisclosed, may be combined in any combination, except combinations whereat least some of such features and/or steps are mutually exclusive. Eachfeature disclosed in the specification, including the claims, abstract,and drawings, can be replaced by alternative features serving the same,equivalent or similar purpose, unless expressly stated otherwise. Thus,unless expressly stated otherwise, each feature disclosed is one exampleonly of a generic series of equivalent or similar features.

This invention is not limited to particular hardware described herein,and any hardware presently existing or developed in the future thatpermits processing of digital images using the method disclosed can beused, including for example, a digital camera system-L

Any currently existing or future developed computer readable mediumsuitable for storing data can be used, including, but not limited tohard drives, floppy disks, digital tape, flash cards, compact discs, andDVDs. The computer readable medium can comprise more than one device,such as two linked hard drives, in communication with the processor.

Also, any element in a claim that does not explicitly state “means for”performing a specified function or “step for” performing a specifiedfunction, should not be interpreted as a “means” or “step” clause asspecified in 35 U.S.C. § 112.

It will also be understood that the term “comprises” (or it grammaticalvariants) as used in this specification is equivalent to the termincludes” and should not be taken as excluding the presence of otherelements or features.

1. A method for image processing of a digital image comprising pixelshaving characteristics, comprising applying an image processing filteras a function of the characteristics of each pixel to be processed, afirst set of target image characteristics, a first received adjustmentparameter associated with the first set of target image characteristics,a second set of target image characteristics, and a second receivedadjustment parameter associated with the second set of target imagecharacteristics.
 2. The method of claim 1, where either the first set oftarget image characteristics, or the second set of target imagecharacteristics, or both, are received.
 3. The method of claim 1 whereinthe image processing filter is a noise reduction filter, a sharpeningfilter, or a color change filter.
 4. The method of claim 1 furthercomprising receiving one or more third sets of target imagecharacteristics, and one or more third adjustment parameters, each ofthe third adjustment parameters being associated with one of the thirdsets of target image characteristics, and wherein the application of theimage processing filter is also a function of the one or more third setsof target image characteristics, and the associated third adjustmentparameters.
 5. The method of claim 1 where either, or both, of thereceived adjustment parameters is an opacity parameter or a luminosityparameter.
 6. The method of claim 1 further comprising the step ofproviding a graphic user interface for receiving the first set of targetimage characteristics, the second set of target image characteristics,the first adjustment parameter, and the second adjustment parameter. 7.The method of claim 6, where the graphic user interface for receivingeither of the adjustment parameters comprises a slider.
 8. The method ofclaim 1 wherein the first set of target image characteristics, or thesecond set of target image characteristics, comprises an imagecoordinate, a color, or an image structure.
 9. (canceled)
 10. The methodof claim 6, where the graphic user interface comprises indiciarepresenting target image characteristics.
 11. The method of claim 6,where the graphic user interface comprises a tool to determine the pixelcharacteristics of an image pixel.
 12. The method of claim 1 furthercomprising the step of providing camera-specific default settings.
 13. Agraphic user interface embodied on a computer-readable medium forexecution on a computer for image processing of a digital image, thedigital image comprising pixels having characteristics, comprising: afirst interface to receive a first set of target image characteristics;a second interface to receive a second set of target imagecharacteristics; a third interface to receive a first adjustmentparameter associated with the first set of target image characteristics;and a fourth interface to receive a second adjustment parameterassociated with the second set of target image characteristics.
 14. Thegraphic user interface of claim 13, further comprising a fifth interfacecomprising indicia representing the first set of target imagecharacteristics, and a sixth interface comprising indicia representingthe second set of target image characteristics.
 15. The graphic userinterface of claim 13, further comprising a tool to determine the pixelcharacteristics of an image pixel.
 16. The graphic user interface ofclaim 13, where the third interface and the fourth interface eachcomprise a slider.
 17. A system for image processing of a digital image,the digital image comprising pixels having characteristics, comprising:a processor, a memory in communication with the processor, and acomputer readable medium in communication with the processor, thecomputer readable medium having contents for causing the processor toperform the steps of: receiving a first set of target imagecharacteristics; receiving a first adjustment parameter associated withthe first set of target image characteristics; receiving a second set oftarget image characteristics; receiving a second adjustment parameterassociated with the second set of target image characteristics;determining for each pixel to be processed, the correspondence betweenthe characteristics of that pixel, the first set of target imagecharacteristics, and second set of target image characteristics; andprocessing the digital image by applying the image processing filter asa function of the determined correspondence, the first receivedadjustment parameter, and the second received adjustment parameter. 18.The system of claim 17, the computer readable medium further havingcontents for causing the processor to perform the steps of receiving oneor more third sets of target image characteristics, and one or morethird adjustment parameters, each of the third adjustment parametersbeing associated with one of the third sets of target imagecharacteristics, and the processing step further comprising applying theimage processing filter as a function of the one or more third sets oftarget image characteristics, and the one or more associated thirdadjustment parameters.
 19. The system of claim 17, further comprising aset of camera-specific default instructions embodied on acomputer-readable medium for execution on a computer.
 20. A set ofcamera-specific default instructions embodied on a computer-readablemedium for execution on a computer for image processing of a digitalimage, using the method of claim
 1. 21. A set of camera-specific defaultinstructions for setting the state of the graphic user interface ofclaim 13, embodied on a computer-readable medium for execution on acomputer. 22-31. (canceled)